Feature: add dynamic interval support. #652
Conversation
Fix pod count accuracy by applying annotation filtering to dynamic interval calculations and update tests to validate correct behavior.
|
|
||
| pods = filterByAnnotations(pods, c.Annotations) | ||
|
|
||
| podCount := len(pods) |
There was a problem hiding this comment.
The logic of finding all possible target pods (after filtering) should pretty much match the logic of Candidates() (https://github.com/linki/chaoskube/blob/master/chaoskube/chaoskube.go#L214). Let's try to re-use it.
There was a problem hiding this comment.
The Candidates() filters out too much, which gives a false number to calculate the dynamic interval.
For instance, filterByMinimumAge or filterByOwnerReference, which do not make sense for the calculation.
Therefore I recreated the list by filtering only the relevant pods to calculate the interval
chaoskube/chaoskube.go
Outdated
| pods = filterByOwnerReference(pods) | ||
| c.Logger.WithFields(log.Fields{ | ||
| "count": len(pods), | ||
| }).Debug("Final pod count after owner reference filtering") |
There was a problem hiding this comment.
We could think about moving the logging to the end of each filterBy* function instead of having it here.
|
Closing this PR to retarget it to a frozen branch. |
2 participants
Dynamic Interval
Add a dynamic interval feature that automatically adjusts the time between pod terminations based on the number of candidate pods in your cluster. This helps ensure appropriate chaos levels in both small and large environments.
How it works
With dynamic interval enabled, chaoskube will calculate the interval between pod terminations using the following formula:
Where:
totalWorkingMinutes= 10 days * 8 hours * 60 minutes = 4800 minutes (we assume that all pods should be killed during 2 work weeks)factoris the configurable dynamic interval factorThe dynamic interval factor lets you control the aggressiveness of the terminations:
factor = 1.0: Standard interval calculationfactor > 1.0: More aggressive terminations (shorter intervals)factor < 1.0: Less aggressive terminations (longer intervals)Example scenarios